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ON EVALUATING COMPLIANCE WITH AIR POLLUTION LEVELS 


"NOT TO BE EXCEED D MORE THAN ONCE PER YEAR" 
by Harold r Neustadler and Sloven M. Sidik 
Lewis Research Center 

National Aeronautics and Space Administration 
Cleveland, Ohio 

ABSTRACT 

This study considers the adequacy of currently practiced monitoring, 
and data reduction techniques for assessing compliance with 24 -hour Air 
Quality Standards (AQS) not to be exceeded more than once per year. 
Consider as an example the present situation for suspended particulates. 
The federal AQS in part mandates a limit of 260 micrograms per cubic 
meter - maximum 24 -hour concentration not to be exceeded more than 
once per year, while EPA guideline documents state that adequate cover- 
age may be maintained with sampling frequencies of from every third 
day to once every sixth day. Because this AQS does not limit the pollution 
load for the worst day of each year, the estimated value one seeks for 
comparison with this AQS is that of the second most polluted (SMP) day. 
Since one has only (typically) 60 to 120 measured values, it is necessary 
to estimate SMI’ in order to determine compliance/noncompliance with 
this AQS which is in terms of the 364th of 365 ranked values. 

We quantitatively consider the inherent variability of estimating SMP 
levels from order statistics for the normal distribution lost on samples of 
30 to 365 measurements. In addition, we consider the validity of: 
assumption of independence between observations: the interchange of 
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exponentiation and expectation operations in extending normal order 
statistics to the log normal distribution; and the substitution of an esti- 
mated-maximum ’’design value” for the SMP value specified in the AQS. 

With regard to the four points discussed above, we conclude 

(1) For typical less than daily sampling (i. e. . BO to 120 24 -hour 
samples per year) the deviation from independence of the data set should 
not be substantial. 

(2) The interchange of exponentiation and expectation operations in 
the EPA data reduction model, underestimates the second highest level 
by about 4 to 8 percent for typical Q values. 

(3) Estimates of the second highest pollution level have associated 
with them a large statistical variability arising from th'* finite size of 
the sample. The 0. 95 confidence interval ranges from ±40 percent for 
120 samples per year to ±84 percent for 30 samples per year. 

(4) The design value suggested by EPA for abatement and/or control 
planning purposes typically gives a margin of safety of 60 to 120 percent. 

INTRODUCTION 

This paper is addressed to various statistical considerations from using 
guidelines for air quality surveillance in conjunction with a mathematical model 
(EPA-MM) as presented in U. S. Environmental Protection Agency (EPA) 
documents (refs. 1 and 2). The point of view we take is that the EPA Air 
Quality Standards (AQS) represent conditions which must be made to exist in 
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Lite ambient environment. The statistical techniques developed should serve 
as tools for measuring’ how close one is to achieving the desired quality of 
air- We shall show that the sampling frequency recommended by EPA is 
inadequate to meet these objectives when the standard is expressed as a level 
not to be exceeded more than once per year and sampling frequency is once 
every three days or less frequent. This inadequancy came to our attention 
in the course of a joint Air Pollution study between NASA Lewis Research 
Center and the City of Cleveland, Ohio Air Pollution Control Division. 

For clarity let us consider as an example the present situation for sus- 
pended particulates, although the same considerations apply to other pollutants 
as well. 

The standards in part are expressed (ref. 3) as 

(a) Maximum of 75 micrograms per cubic meter - annual geometric mean. 

(b) 260 micrograms per cubic meter - maximum 24-hour concentration 
not to be exceeded more than once per year. 

With regard to monitoring it is stated that (ref. 1) adequate coverage may 
be maintained with intermittent sampling . Suggested sampling frequencies are 
from every third day to once every sixth day. 

It is obvious that part (a) of the AQS can best be estimated by the mean of 
1 

the sample data. For notational purposes we refer to part (b) of the AQS as 

^The reliability (variability) of such an estimate is considered in reference 4. 

The correct expression is given in their equation (1). The use of equation(2) 

is an error arising from a misconception of the statistical model as is made 

clear later in this paper aod elsewhere (ref. 5) . 
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MAX2 and observe that MAX2 does nut put any direct limit on how polluted the 
ambient air may be on one day of each year. Thus, the estimated value one 
seeks for comparison with MAX2 is that of the second must polluted (SMP) day. 
It is considerably more difficult to estimate SMP in order to determine 
compliance/noncompliance with MAX2 which is in terms of 385 ranked values, 
since one has only (typically) 30 to 120 measured values. 

In the framework of the above introduction we will consider the inherent 
variability of estimating SMP levels from samples of 30 to 335 measurements. 
In addition we will consider three other points that have appeared recently 
(refs. 3 and 7) concerning EPA-MM. Patel (ref. 6) has questioned two aspects 
of the analysis, namely the assumption of independence between observations 
and the interchange of the exponentiation and expectation operators (IEE) . 
Larsen in his response (ref. 7) minimizes the practical impact of these 
criticisms by emphasizing an additional (intentional) irregularity, namely, the 
substitution of a maximum "design value" for the next -to -the -highest value 
specified in the national ambient air quality standards (AQS). 

STATISTICAL CONSIDERATIONS 

We feel that it is necessary for a distinction to be made between the legal 
and statistical aspects of viewing the data. From the statistical viewpoint one 
assumes each successive day represents an independent pollutant level from a 
single population of potential pollutant levels (theoretically infinite in size). It 
is this assumption of independence that Patel questions. We realize that this 
may not be strictly valid. In fact, a time series representation of the data 
is the preferred analysis. However, the representation of the pollution process 
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as a time series is nut always a feasible approach, especially for small, 
irregularly spaced sample sets. We intend to show that even though approxi- 
mate independence can be achieved, the estimate of MAX 2 is too variable 
for practical use. 

Some assumption is then made about the distribution of the pollutant levels. 
The standard, and empirically most satisfying' assumption is that the log- 
normal distribution (ref. 8) is an adequate representation. Statistically speak- 
ing the set of levels that would be obtained by actually monitoring on each of 
335 days of the year is a single random sample of size 385 from an infinite 
population of possible levels. Sampling every second (third, fourth, etc.) day 
instead of every day only reduces the size of the sample. It does not introduce 
any new statistical concepts of populations. For the purpose of studying pollu- 
tion, designing an abatement or control program, analyzing health effects etc. 
this is the appropriate concept. 

Two possible model errors are that the distribution of levels may not be 
as claimed or that the distribution may change with time. Further errors 
would arise from the fact we are only obtaining a finite (even if it be of size 
385) random sample from an infinite population. In this sense even daily 
sampling does not represent perfect sampling and complete knowledge because 
of meteorological, economic etc. variability from year to year. 

The emphasis and outlook must change when one takes the legal aspect of 
pollution monitoring into account. If, for example, the air is monitored every 
10th day, and one finds that the mean for this sample is 55 and the highest 
measured value is 255, then it is very likely that the actual SMP for the year 
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exceeds 280. This is not legally equivalent to showing Unit SMP did in fad 
exceed 280. Thus, in a sense, sampling every day is a legal optimum. 

As stated previously, the point of view we take is that the legal AQS repre- 
sents conditions which must be made to exist in the ambient environment. 

The statistical techniques developed should serve as tools for measuring how 
close one is to achieving the desired quality of air. Thus we are basically 
concerned with the reliability of estimates of the mean pollution level and the 
334th of 365 ranked observations considered as a sample from an infinite 
population, We are also concerned with the reliability (variability) of these 
estimates. 

If the previously mentioned assumptions, namely independence and log 
normality, are made, it is possible to proceed. (We note again that if a time 
series representation were more practical it could also provide such esti- 
mates.) EPA has suggested a mathematical model which is partially graphical 
(EPA-MM). In our experience as part of a cooperative program with Cleveland, 
Ohio (ref. 9) we have found it simpler to use an analytical formula which'is 
presented Mer as equation (2). (The implications of using eq. (2) as well as 
the motivation for the use of this model are discussed in some detail 
in appendix A of the NASA publication NASA TN D-7527. ) 

Both of these formulations in effect (1) transform the log-normally dis- 
tributed data set into a normally distributed data set, (2) compute the mean 
and standard deviation of the sample, (3) use these two values obtained from the 
sample as estimates for the population parameters, (4) compute the expected 
384th of 385 ranked observations from that population, and (5) estimate the 
actual SMP as the exponential of the value obtained in (4). It is this sequence 


# 75-52 



7 


Unit is questioned by Patel who correctly points oul Hull exponentiating a value 
estimated from a normal set is not the same as estimating the expected next 
to the largest value directly from equivalent log-normal sets of size 300. It 
should be explained that this use of normal order statistics did not result 
from ignorance of the point raised by Patel, but rather from the fact that the 
order statistics are well known for the normal distribution (ref. 10), Only 
recently have they even been evaluated for very small sample sizes of log- 
normally distributed sets (ref. 11). 

As mentioned earlier, EPA (ref. 7) suggests that for control and abatement 
planning one should estimate the expected 365th of 365 ranked values, rather 
than the expected 364th value. This is designated as a "design value" to 
compensate for variability from sample to sample. However, there is no dis- 
cussion of the magnitude or adequacy of such a correction factor. Nor is the 
approach of any assistance in determining actual compliance with AQS. This 
"design value" is discussed quantitatively in a later section. 

INDEPENDENCE OF DATA 

Having briefly summarized the points of interest and the pertinent statis- 
tical considerations we will now make some quantitative remarks. The 
assumption of independence was questioned intuitively by Patel. He also demon- 
strated for National Air Surveillance Network - Continuous Air Monitoring 
Program (CAMP) data (refs. 2 and 6) that a criterion for independence based 
on the following test of the ratios of variances is not met. 

If Xp . . ., x n are all. independent and identically distributed random 
variables with variance or 2 , then m x/n has variance cr 2 /n. Thus the 
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variance of data accumulated using; x-hour averaged data (V ) divided by the 
variance of data accumulated using y-hour averaged data (V ) should be about 

V 2/ 

V 2/ 
y o /y 

We have applied the same test to data amassed over six years by the Air 
Pollution Control Division of Cleveland, Ohio and in general do n< ! find the 
deviation from independence to be at all substantial. The results are listed 
in table I -III. Table I is for total suspended particulate and table II is for 
nitrogen dioxide. Table III is for sulfur dioxide and also lists the correspond- 
ing results obtained by Patel for CAMP data. At the bottom of each table we 
also list the expected values for fufly independent sets. Overall, the devi- 
ations from anticipated values, assuming independence, are not substantial 
and nowhere do our calculated values approach those obtained by Patel. 

A further check on the independence of the data may be made as follows . 

If the successive data values (x^, Xg, .... x n ) are truly independent, then 
there should be zero correlation between pair of singly offset values (e.g. 
(x-^Xg), (xg.xg) . . .) and similarly for doubly offset pairs (e.g. (x^,Xg), 
(xgjX^). Also shown on the right side of tables I-III are the results of these 
calculations. Again, there is some evidence of small positive correlations, 
but not sufficiently so to be considered serious. 

Intuitively, this appears lo be a reasonable result. The measurements 
used in CAMP were taken every five minutes and it is to be expected that 
successive measurements should be related. On the other hand, the time 
interval between successive measurements in the data we analyzed has 
typically been from three to six days. 
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In view of the meteorological variability of the region it is not very prob- 
able that successive data points so separated in time would be substantially 
correlated. This naturally is satisfying in terms of the applicability of the 
statistical model. However, it will be seen later that Hus three to six day 
interval between measurements is not without its cost, in that the uncertainly 
it introduces into SMP estimates is quite large. 

INTERCHANGE OF EXPONENTIATION AND EXPECTATION (IEE) 

The problem with preceding the exponentiation operation by the expectation 
operation is portrayed clearly with an artifieal example by Patel (ref. 3). 

To determine how much effect this might have on the analysis of air 
pollution data we performed a computer simulation of the correct procedure 
and compared it with the estimation based on normal order statistics. The 
experiment was a Monte-Carlo (MC) simulation provided by sampling from a 
nearly infinite (2 - 1) standard normal set (mean, g = 0; standard deviation, 

cr - 1) from which 365 values were randomly selected and exponentiated. This 
set of 365 values was then ranked and the 364th value, corresponding to SMP, 
was located. This was repeated 3800 times and the mean and standard 
deviations were obtained for these 3800 SMP values. The mean SMP was 
14.27. This compares with 13.87 based on normal order statistics where 

SMP = e (M + 2* 63 o) (!) 

and 

^ = 0 

o - 1 

2.63 is the normal order statistic for the 364th of 365 ranked values 
(ref. XI) 
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In practice we use 

SMP - exp(m + 2.63 s) (2) 

ni - sample mean of the normal variates 
s - sample standard deviation of the normal variates 
6y contrast ln(X4 . 27) - 2.66. To approximately assess the impact of this 
difference we can take the ratio of the estimate from the method using normal 
order statistics to our method using the Monte Carlo equivalent order statistic 
and find 


t (u * 2.63 a) 
Jfj » 2. 66 cr) 


e -0.03 a, x _ 0 03 


(a - 3). 


For typical air quality doia (ref. 10) 1.3 C cr^ 2.7. Thus the error in invert- 
ing the sequence of operations results in underestimating the actual value by 
about 4 to 8 percent 


"DESIGN VALUE" SAFETY FACTOR 


For planning purposes the EPA suggests the use of the expected annual 
maximum level as opposed to the expected annual SMP in order to compen- 
sate for the year to year fluctuations (refs. 2 and 7) and the underestimate in- 
herent in EPA-MM itself (ref. 7). The suggested corresponding normal order 
statistic is 2.94. The margin of safety introduced by this suggestion can be 
estimated as we did above and indicates a margin of safety over using SMP 
of e ’ . For a = 1.5 this is a 60 percent overestimate while for c -1.5 

this is a 60 percent overestimate while for a = 2.5 it is a 117 percent over- 
estimate. This is obviously adequate to compensate for the IEE error. 
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SMP VARIABILITY 


In order to get Home information on the variability of the eHtimalod 
SMP value a further series of Monte-Carlo computer experiments were per- 
formed. Similar to the experiment described above we sampled ti pseudo 
random observations from a normal distribution with mean zero and standard 
deviation 1. For each set of n observations we used equation ( 2 ) to esti- 
mate SMP and recorded the value. This was repealed thousands of tunes 
for each n(n ~ 30. 60. 90, 120. 365). Based on this we obtained the results 
listed in table IV and which are also shown graphically in figures 1 and 3. 

The variability is shown in the last column of table IV which indicates the 
range of values expressed as a percent of the mean SMP required Lo include 
95 percent of all the calculated SMP's. The first 5 rows give the results of 


this experiment. The next row shows the results of the Monte-Carlo expert 


meat described earlier with the values obtained from actual rankings. 


For 


samples of size 365 the calculations show the variability to be greater in a 
straight-forward ranking than when calculated from equation (2). This is 
quite reasonable as the latter gives more weight to the entire sample set 
through m and s. The calculations also show t hat the smaller the sample 
size, the larger the expectation of SMP. This is a consequence of the prop- 
erties of the distribution of s. We also dicplay the expected SMP values in 
figure 1 and the variability in figure 2 as plots against n. The value of 14.27 
obtained from the 365 ranked values appears to be the asymptote for the cal- 
culated values. 

The present federal monitoring schedule requires a minimum ol 
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00 samples per year (ref, 1) which gives a variability «»t Sti pcrt-oni at 
the 0. 05 confidence level. This is well bounded by the MPA t value. " 
However, in attempting to assess compliance in any given year *»r trends 
over a few years (as opposed to control planning) it appears that where a 
large time interval exists between measurements (which makes the sample 
set independent) the utility of estimates of BMP is marginal because of the 
large* variability, 

CONCLUSION 

With regard to the four points discussed above we conclude 

(1) For typical less than dally sampling (i. e, , GO to 120 24 -hour 
samples per year) the deviation from independence of the data set should 
not be substantial. 

(2) The interchange of exponentiation and expectation operations In 
the EPA data reduction model, underestimates the second highest level 
by about 4 to 8 percent for typical a values, 

(3) Estimates of tne second highest pollution level have associated 
with them a large statistical variability arising from the finite size of 
the sample. The 0. 95 confidence interval ranges from ±40 percent for 
120 samples per year to ±84 percent for 30 samples per year. 

(4) The design value suggested by EPA for abatement and/or control 
planning purposes typically gives a margin of safely of GO to 120 percent. 
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TABLE 1. - STATISTICS FOR MEASURING INDEPENDENCE OFTSP OBSERVATIONS. LISTED ARE 
THE OBSERVED RATIOS VARIANCES FOR X-HR AVERAGES OPPOSED TO Y-HR AVERAGES AND 
THE OBSERVED CORRELATIONS FOR SINGLE AND DOUBLE OFFSETS 
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TABLE II. - STATISTICS FOR MEASURING INDEPENDENCE OF N0 2 OBSERVATIONS. LISTED ARE 
THE OBSERVED RATIOS OF VARIANCES FOR X-HR AVERAGES OPPOSED TO Y-HR AVERAGES 
AND THE OBSERVED CORRELATIONS FOR SINGLE AND DOUBLE OFFSETS 
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TABLE ill. - STATISTICS FOR MEASURING INDEPENDENCE OF SO? OBSERVATIONS. LISTED ARE 
THE OBSERVED RATIOS OF VARIANCES FOR X-HR AVERAGES OPPOSED TO Y-HR AVERAGES 
AND THE OBSERVED CORRELATIONS FOR SINGLE AND DOUBLE OFFSETS 
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TABLE *V. “ ESTIMATED SMP LEVELS FROM 
MONTE-CARLO EXPERIMENT 


Method 



n 

Mean SMP 
value 

# Samples 



95 Percent j 
confidence j 
bounds 

e (m + 2.63 s) 

30 

15,10 

3400 

±84% 

m and s calculated 

60 

14. 70 

3000 

±56% 

from Monte-Carlo 

90 

14. 47 

3000 

+44% 

generated data sets 

120 

14.40 

1400 

+40% 


365 

14.33 

1500 

±21% 

Ranking of Monte- 
Carlo generated 
data sets 

365 

14. 27 

3800 

±58% 

.. . 
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